feat(block_service): add JuiceFS support as block storage backend#2384
feat(block_service): add JuiceFS support as block storage backend#2384ruojieranyishen wants to merge 5 commits intoapache:masterfrom
Conversation
…ache#2383) apache#2383 This commit adds support for JuiceFS as a block service backend in Pegasus. JuiceFS provider is recognized by the jfs:// prefix pattern and reuses the existing hdfs_service implementation for file operations.
acelyc111-bot
left a comment
There was a problem hiding this comment.
Review: JuiceFS block storage support
Summary: Adds JuiceFS as a new block storage backend by reusing the HDFS service implementation. Detects JuiceFS providers via jfs:// prefix pattern.
What's good:
- Clever approach — leverages existing HDFS service since JuiceFS provides HDFS-compatible interface
- Provider detection logic (
is_juicefs_provider) validates format correctly (prefix,@, non-empty host) - Clean separation of concerns for args handling between JuiceFS and standard providers
Issues / Suggestions:
-
Hardcoded jar version in pack_server.sh —
juicefs-hadoop-1.3.1.jaris downloaded withwget -Nat build time. This will fail in air-gapped build environments. Consider making the version configurable or bundling it as a dependency. -
jfs://prefix is a magic string — Extract to a constant (e.g.,JUICEFS_PROVIDER_PREFIX) alongsideBLOCK_SERVICE_JUICEFS. -
No unit tests for
is_juicefs_provider— The validation logic is simple but worth testing edge cases likejfs://(no user),jfs://user@(empty host),jfs://@host(empty user),dfs://user@host(wrong prefix). -
Parser throws away parsed components — The function validates
user@hostformat but then reconstructs args from the raw provider string. The parsedusercould be validated further (e.g., non-empty after prefix). -
Test script uses example placeholder — The
package_dir="example: ..."inrun.shlooks like documentation rather than a working config. Should this be a parameter or env var default?
Verdict:
#2383
This commit adds support for JuiceFS as a block service backend in Pegasus. JuiceFS provider is recognized by the jfs:// prefix pattern and reuses the existing hdfs_service implementation for file operations.
Todo